31 research outputs found
Improving Optimization Bounds using Machine Learning: Decision Diagrams meet Deep Reinforcement Learning
Finding tight bounds on the optimal solution is a critical element of
practical solution methods for discrete optimization problems. In the last
decade, decision diagrams (DDs) have brought a new perspective on obtaining
upper and lower bounds that can be significantly better than classical bounding
mechanisms, such as linear relaxations. It is well known that the quality of
the bounds achieved through this flexible bounding method is highly reliant on
the ordering of variables chosen for building the diagram, and finding an
ordering that optimizes standard metrics is an NP-hard problem. In this paper,
we propose an innovative and generic approach based on deep reinforcement
learning for obtaining an ordering for tightening the bounds obtained with
relaxed and restricted DDs. We apply the approach to both the Maximum
Independent Set Problem and the Maximum Cut Problem. Experimental results on
synthetic instances show that the deep reinforcement learning approach, by
achieving tighter objective function bounds, generally outperforms ordering
methods commonly used in the literature when the distribution of instances is
known. To the best knowledge of the authors, this is the first paper to apply
machine learning to directly improve relaxation bounds obtained by
general-purpose bounding mechanisms for combinatorial optimization problems.Comment: Accepted and presented at AAAI'1
Improved Peel-and-Bound: Methods for Generating Dual Bounds with Multivalued Decision Diagrams
Decision diagrams are an increasingly important tool in cutting-edge solvers
for discrete optimization. However, the field of decision diagrams is
relatively new, and is still incorporating the library of techniques that
conventional solvers have had decades to build. We drew inspiration from the
warm-start technique used in conventional solvers to address one of the major
challenges faced by decision diagram based methods. Decision diagrams become
more useful the wider they are allowed to be, but also become more costly to
generate, especially with large numbers of variables. In the original version
of this paper, we presented a method of peeling off a sub-graph of previously
constructed diagrams and using it as the initial diagram for subsequent
iterations that we call peel-and-bound. We tested the method on the sequence
ordering problem, and our results indicate that our peel-and-bound scheme
generates stronger bounds than a branch-and-bound scheme using the same
propagators, and at significantly less computational cost. In this extended
version of the paper, we also propose new methods for using relaxed decision
diagrams to improve the solutions found using restricted decision diagrams,
discuss the heuristic decisions involved with the parallelization of
peel-and-bound, and discuss how peel-and-bound can be hyper-optimized for
sequencing problems. Furthermore, we test the new methods on the sequence
ordering problem and the traveling salesman problem with time-windows (TSPTW),
and include an updated and generalized implementation of the algorithm capable
of handling any discrete optimization problem. The new results show that
peel-and-bound outperforms ddo (a decision diagram based branch-and-bound
solver) on the TSPTW. We also close 15 open benchmark instances of the TSPTW.Comment: 50 pages, 31 figures, published by JAIR, supplementary materials at
https://github.com/IsaacRudich/ImprovedPnB. arXiv admin note: substantial
text overlap with arXiv:2205.0521
Verification of interlocking systems using statistical model checking
In the railway domain, an interlocking is the system ensuring safe train
traffic inside a station by controlling its active elements such as the signals
or points. Modern interlockings are configured using particular data, called
application data, reflecting the track layout and defining the actions that the
interlocking can take. The safety of the train traffic relies thereby on
application data correctness, errors inside them can cause safety issues such
as derailments or collisions. Given the high level of safety required by such a
system, its verification is a critical concern. In addition to the safety, an
interlocking must also ensure that availability properties, stating that no
train would be stopped forever in a station, are satisfied. Most of the
research dealing with this verification relies on model checking. However, due
to the state space explosion problem, this approach does not scale for large
stations. More recently, a discrete event simulation approach limiting the
verification to a set of likely scenarios, was proposed. The simulation enables
the verification of larger stations, but with no proof that all the interesting
scenarios are covered by the simulation. In this paper, we apply an
intermediate statistical model checking approach, offering both the advantages
of model checking and simulation. Even if exhaustiveness is not obtained,
statistical model checking evaluates with a parametrizable confidence the
reliability and the availability of the entire system.Comment: 12 pages, 3 figures, 2 table
Combining Reinforcement Learning and Constraint Programming for Combinatorial Optimization
Combinatorial optimization has found applications in numerous fields, from
aerospace to transportation planning and economics. The goal is to find an
optimal solution among a finite set of possibilities. The well-known challenge
one faces with combinatorial optimization is the state-space explosion problem:
the number of possibilities grows exponentially with the problem size, which
makes solving intractable for large problems. In the last years, deep
reinforcement learning (DRL) has shown its promise for designing good
heuristics dedicated to solve NP-hard combinatorial optimization problems.
However, current approaches have two shortcomings: (1) they mainly focus on the
standard travelling salesman problem and they cannot be easily extended to
other problems, and (2) they only provide an approximate solution with no
systematic ways to improve it or to prove optimality. In another context,
constraint programming (CP) is a generic tool to solve combinatorial
optimization problems. Based on a complete search procedure, it will always
find the optimal solution if we allow an execution time large enough. A
critical design choice, that makes CP non-trivial to use in practice, is the
branching decision, directing how the search space is explored. In this work,
we propose a general and hybrid approach, based on DRL and CP, for solving
combinatorial optimization problems. The core of our approach is based on a
dynamic programming formulation, that acts as a bridge between both techniques.
We experimentally show that our solver is efficient to solve two challenging
problems: the traveling salesman problem with time windows, and the 4-moments
portfolio optimization problem. Results obtained show that the framework
introduced outperforms the stand-alone RL and CP solutions, while being
competitive with industrial solvers
Verification of railway interlocking systems and optimisation of railway traffic
Since the dawn of the nineteenth century, development of railway systems has taken a huge importance in many countries. Over the years, the number of trains, the number of tracks, the complexity of networks increase and are still increasing. Directing trains on efficient routes, stopping and cancelling them are some actions that railway operators must take in their everyday life in order to regulate the traffic. However, with its continual growth, the consequences of such actions become rapidly hard to predict. Bad decisions can lead to disastrous situations such as accidents or, in the best cases, to unnecessary delays leading to financial losses. Decisions and actions that could be taken manually in the past are now hard combinatorial problems that require computer based methods for their solving. In this context, the need of a reliable and efficient railway traffic management is crucial. Like any transportation system, three aspects must be considered: safety, availability and fluidity. Safety and availability belong to verification engineering while fluidity is related to optimisation. A plethora of research on this field already exist. However, most of it suffers of a lack of scalability. They can only be used for small or medium stations. This thesis presents innovative approaches for tackling this problem. For each aspect, we propose a method, that is feasible in practice for stations of any size. Concretely, verification of safety is performed with a dedicated algorithm while availability is verified with Statistical Model Checking. Fluidity optimisation is carried out with Constraint Programming. The performance of these methods are analysed through three stations of the Belgian railway network.(FSA - Sciences de l'ingénieur) -- UCL, 201
Peel-And-Bound: Generating Stronger Relaxed Bounds with Multivalued Decision Diagrams
Decision diagrams are an increasingly important tool in cutting-edge solvers for discrete optimization. However, the field of decision diagrams is relatively new, and is still incorporating the library of techniques that conventional solvers have had decades to build. We drew inspiration from the warm-start technique used in conventional solvers to address one of the major challenges faced by decision diagram based methods. Decision diagrams become more useful the wider they are allowed to be, but also become more costly to generate, especially with large numbers of variables. We present a method of peeling off a sub-graph of previously constructed diagrams and using it as the initial diagram for subsequent iterations that we call peel-and-bound. We test the method on the sequence ordering problem, and our results indicate that our peel-and-bound scheme generates stronger bounds than a branch-and-bound scheme using the same propagators, and at significantly less computational cost